In public clouds such as Amazon's S3, the delay for a single read or write operation for small objects (e.g., less than or equal to 1 Kbyte) can be hundreds of milliseconds of delay, while for medium size objects (e.g., >1 Mbyte) delays become in the order of seconds at the 99th and 99.9th percentiles. For cascaded operations where one transaction needs many reads and writes to the same storage facility, these delays can be unacceptably high. For video content that consists of many megabytes, how to use S3 type storage as the video archive, while attaining small startup delays and no pauses for video playback also becomes a critical issue.
In storage systems such as RAID, distributed storage solutions based on DHTs, content distribution networks, the system designer has the control over how to stripe the data, where to place each data part, replication (coded/uncoded) locations, etc. A representative block diagram for these existing systems is shown in FIG. 1. Referring to FIG. 1, a storage controller with full knowledge of storage elements performs parallel reads and writes to storage devices/systems to increase throughput. This storage controller also employs FEC encoding for reliability and FEC decoding for recovering data if some storage elements fail. The storage controller can have both software and hardware based parts for optimized integrated solution.
For video communication, applying forward error correction and using parallel weakly correlated paths to send subsequent streaming packets between a source-destination pair are well-known techniques. A representative illustration for this is shown in FIG. 2, where a source node uses three disjoint paths to send packets to its destination node. The source packets are expanded with parity packets for extended protection against packet losses. As late packets in this context are equivalent to packet losses, one can treat the scenario as delay-sensitive application of FEC and path diversity. In these set ups, however, disjoint paths are visible to the source as different interfaces or as a whole topology.